{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. 线性回归概念"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "线性回归模型："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "    y = w_1x_1+w_2x_2+...+w_nx_n+b\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中，$n$为影响因素的个数，比如：房价预测问题，影响因素有地段、日期、面积，此时$x_1$表示地段、$x_2$表示日期、$x_3$表示面积"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "损失函数："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "    loss = \\frac{1}{2n} \\sum{(y_i - y)^2}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通常的均方误差损失函数只要$\\frac{1}{n}$即可，这里多除个2是为了求导的方便"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对损失函数求导："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![image-20240426173613646](https://zyc-learning-1309954661.cos.ap-nanjing.myqcloud.com/machine-learning-pic/image-20240426173613646.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里的`X`是一个`m×n`矩阵，列数表示影响因素的个数，行数表示数据的条数。比如：房价预测考虑地段、面积、日期三个因素，同时有100条数据，此时`X`就是一个`100×3`的矩阵。由于引入偏置`b`，所以要在每一条数据最后加个`1`。\n",
    "\n",
    "`w`是一个`n`维列向量。\n",
    "\n",
    "`X·w`的结果是一个`m`维列向量，对这个列向量做内积并取平均，即可得到MSE。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. 从零实现线性回归"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 205,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import matplotlib.pyplot as plt\n",
    "import random"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1 生成人造数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 206,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(torch.Size([1000, 2]), torch.Size([1000, 1]))"
      ]
     },
     "execution_count": 206,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def synthetic_data(w, b, num_example):\n",
    "    # 随机生成X，用X计算得到标签y，y = Xw + b + 噪声\n",
    "    # 噪声的目的是：给原本的X和y之间绝对的线性关系中添加非线性因素\n",
    "    X = torch.normal(0, 1, (num_example, len(w)))\n",
    "    y = torch.matmul(X,w) + b\n",
    "    y += torch.normal(0, 0.01, y.shape)\n",
    "\n",
    "    return X, y.reshape((-1,1))\n",
    "\n",
    "w = torch.tensor([2.0,-3.4])    # 这里w的元素必须是float，即要写成小数，[2,-3]就不行，矩阵乘法不支持float和long矩阵相乘\n",
    "b = 4.2\n",
    "features, labels = synthetic_data(w, b, 1000)\n",
    "\n",
    "features.shape, labels.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 207,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x245e4e01270>"
      ]
     },
     "execution_count": 207,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAGdCAYAAAAvwBgXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAABsM0lEQVR4nO29e5BU5Z3//wGEERGGQWbAyUwDA8NACGKjkEDMl5uK1LeSyLjK1m5FTSxruXiJZLNAvpVY1n7zBTaWxFgqbiVitn5hJYlM3E2KqIWApjBya1wvMIKgPTACMzDM4CQZlOnfH/g0p59+rufS55zu96uKKqYv5zzn0ud5P59rn0wmkyEAAAAAgBDoG/YAAAAAAFC6QIgAAAAAIDQgRAAAAAAQGhAiAAAAAAgNCBEAAAAAhAaECAAAAABCA0IEAAAAAKEBIQIAAACA0Lgs7AGo6O3tpdbWVho8eDD16dMn7OEAAAAAwIBMJkPnzp2j6upq6ttXbfOItBBpbW2l2trasIcBAAAAABe0tLRQTU2N8jORFiKDBw8moosHMmTIkJBHAwAAAAATurq6qLa2NjuPq4i0EGHumCFDhkCIAAAAADHDJKwCwaoAAAAACA0IEQAAAACEBoQIAAAAAEIDQgQAAAAAoQEhAgAAAIDQCFSIPP3003TNNddks15mzJhBW7ZsCXKXAAAAAIgRgQqRmpoaWrNmDe3du5f27NlDc+fOpW9+85v07rvvBrlbAAAAAMSEPplMJlPIHQ4bNox+8pOf0D333KP9bFdXF5WXl1NnZyfqiAAAAAAxwWb+LlhBswsXLtBvfvMb6u7uphkzZgg/09PTQz09Pdm/u7q6CjU8AAAAAIRA4MGqb7/9Nl155ZVUVlZGixcvpqamJvriF78o/Ozq1aupvLw8+w99ZgAAAIDiJnDXzPnz5ymdTlNnZyf99re/pZ///Oe0Y8cOoRgRWURqa2vhmgGxIpXuoKPt3TRm+CBKJirCHg4AABQcG9dMwWNEbrzxRho7diw988wz2s8iRgTEjTVbDtD6HUeyfy+eVUcrF0wMcUQAAFB4bObvgtcR6e3tzbF6AFAspNIdOSKEiGj9jiOUSneENCIAAIg+gQarrlq1ihYsWECJRILOnTtHGzdupO3bt9NLL70U5G4BCIWj7d3S1+GiAQAAMYEKkVOnTtGdd95JH3/8MZWXl9M111xDL730Et10001B7haAUBgzfJDV6wAAAAIWIr/4xS+C3DwAkSKZqKDFs+py3DNLZtXBGgIAAAoKVkcEgFJg5YKJNH/SSGTNAACAIRAiAPhMMlEBAQIAAIag+y4AAAAAQgNCBAAAAAChASECAAAAgNBAjAgAEQBl4QEApQqECAAhg7LwAIBSBq4ZAEIEZeEBAKUOhAgAIaIqCw8AAKUAhAgAIYKy8ACAUgdCBIAQYWXhnTQmqxGwCgAoGSBEAAiZlQsm0sJkdfbvzalWWrPlQIgjAgCAwgEhAkDIpNId1JRqzXkNAasAgFIBQgSAkEHAKgiDVLqDNu87BsELQgd1RAAIGQSsgkKD2jUgSsAiAkDIiAJWl8yqQ8AqCATUrgFRAxYRACLAygUTaf6kkSjzDgJH5QrEfQfCAEIEgIiQTFRgIigSotw7CK5AEDUgRAAAwEeiHn/BXIHOMcIVCMIEQgQAAHxCFn8xf9LISE30cAWCKAEhAgAoGYJ2mcQp/gKuQBAVIEQAACVBIVwmiL8AwB6k7wIACkKYBbQKlbKKVGwA7IFFBAAQOGEHcBbSZYL4CwDsgBABsSPKqZEgnygEcBbaZYL4CwDMgRABsSLslXVUibI4i0IAJ1JWAYguECIgNkRhZR1Foi7OohLACZcJANEEwaogNsS5S61fgZr8dsLoG2J7LFEK4EwmKqhxag1ECAARAhYREBuisrK2xS+LhWg740cMFn5W5/Zw68pxeyywRgAAZMAiAmJDlFbWpvhlsZBt59MLvcLPq8TZmi0HaOFTO2n5r9+ihU/tpDVbDhjtf90rzZ6OBdYIAIAIWERArIjbytqvQE3Zdvr362sVhOkmzoa3gojGZnIshQqoLbb9AFDsQIiA2BGn1Ei/3Emq7TROrTEWZ7bCSCRcTMfmpFABtcW2HwBKAbhmQCwIsyqnF/xyJ+m2Y+r2sBVGukBgk2MpVEBtse0HgFIBFhEQKibm7bivPv1yJ5luR3VObetpyATKg/PG0eyGKqNj0Vlh/HJxFKpeSRTqogBQTECIgNAwERjFUjvEL3eSbjsm59RGGMmEy0M3NRiPWWWF8VNkFiqrKq7ZWwBEFbhmQCiYmrfjXDuk0Ni4DGwyWFYumEhNS2fSY3dMoaalM2mFpVCQuZXY+EzG62U/fgvWOGZvARBlArWIrF69mjZv3kwHDx6kgQMH0syZM2nt2rXU0GC+mgLFial5G6tPMSJ3RpAuA68WHZEVZvO+Y8LPehlvobKq4pa9BUCUCVSI7Nixg5YtW0bTpk2jzz77jH7wgx/QzTffTO+99x4NGlTaE0lQxCWl0FRgoEdIPjJ3huycfnS6m1LpjtDPGS9mghKZhcqqilP2FgBRpk8mk8kUamdtbW1UVVVFO3bsoP/1v/6X9vNdXV1UXl5OnZ2dNGTIkAKMMN7ELaiTH++SWXVSs39cBFbQpNIdtPCpnXmvNy2dSclEhbLmRxTvB5t7AAAQH2zm74IGq3Z2dhIR0bBhw4Tv9/T0UE9PT/bvrq6ugoyrGIhjUKdt0KQfxxF3QaNzv7Bzur35FD2+9XDOZ6J4P8DFAQAomBDp7e2l7373u/TVr36VvvSlLwk/s3r1anrkkUcKNaSiIq4phYU0b8fNYiTCxJ2RTFTE6n6Ai0NM3EUzAKYUTIgsW7aM3nnnHfrTn/4k/cyqVato+fLl2b+7urqotra2EMOLPQjqVBNHi5EI05gZ3A/+U0hhwIvmhclq+lp9JUQJKEoKIkTuu+8++v3vf0+vvfYa1dTUSD9XVlZGZWVlhRhS0YGgTjVhWQjcTl6q7/HuDCKida80ExFli4zhfvCXQlrTRKK5KdVKTanWwPcNQBgEKkQymQzdf//91NTURNu3b6cxY8YEubuSB/52OWFYCNxOXibfY2KD/+zjWw/T4ll1NH/SSBo/YjCtvW0y9e/Xt2TuhyCsFibWND/3q6uRE0dLHgAqAhUiy5Yto40bN9KLL75IgwcPphMnThARUXl5OQ0cODDIXZcs8LeLKbSFwK0ryOZ7soZ063ccyRMyjVPllsgwCEIwBGW10FnT/N6viTgO0pKH2BRQaAIVIk8//TQREc2ePTvn9Q0bNtDdd98d5K4ByKOQFiO3riCb75lWl43aCjoIwRBkDJDKmhbEfkWi2XRMXimGgG4QPwIt8Z7JZIT/IEJAWNiUNveCW1eQzfdsJqOj7d2R6GAcVOfaIFsBqEq6B7VfZ1n9hclq4b79Bl2FQVig6R0AAeDWFWTyPafpXLdyZrx+qI2W//qt7N9hrXRlE/T25lOeJtegq8rKrGlBxh4xN2vj1Bq6c8bowC15cUr5BsVFQSur2oLKqiBICuEL9ztrRmQ6ZwXMiC5mzbz07omczzQmq2nz5xkXTlg11qCPx/kdIhJWhmXH4lYcySrOMhYmq2ndoqSrbesoluqwuqq9ANhgM39DiICSxK0vPMxAPt1EwU/47P9H27tzrCGMx+6YYhXE6uacib5DlN91lz8WWzbvOyY8RidBipEg74sw65fEVVSB8IlsiXcAooDbAMOwA/lUpnPeCmIy6TrdB7rJzs05k32naelMKrusb14JenYsbiZbE1dIU6qV7pwxOpDJPKhstULfcygBAMIg0GBVAKKIakKXBXRGIZBPNtl+eqFXWADroU0pIlIHWxJdnOwWPrWTlv/6LVr41E5as+VA3j7cBGWqvjO7oUr4nk1shfNaiY7RZkxRJKx7zhnQHYUAZ1D8wCICSg7ZZKcK6IxCIJ8skLV/P/F6wmkBkK10TS0dboIyVcLJbTAvs9y8fqgtW2mU6NK1Ysf44v7jtOP9dqvxRo2w77mwLYCgdIAQASWHaBIUBXQ6J+So9G5xdtclIqllgeGctETuA5PJjk3+C5PVOZO/TjjI6mGseOFt4X51wWr8xOjEea3Yiv6hTSmr8UaNMO+5YunNBOIBhAgoSXgLwdH2bmFmCZuQo9S7xRkPwkq68yKB4aVuSSrdQU+8eohePdiWfd22+drKBRNpzPBBtOKFt3Nel1WElU10siqyTnhLwbpFSbpzxmhj0RY1wrznwrbGgNICQgSULCYBhs6JOgqBfKoAUCJSWgBEAamyyY4PfmW4CfiUuY5EyCY6k9gOkagSibY4uRfCuudsrTEoCw+8ACECAJmvPsPu5aNaqTILwNH2bvr0Qi/179c3G8ip8veLuvmqanLYroptXAlu3RGia1Us7oUw7jkbawxiSYBXIEQA+JygVp9+rhZVAaCb9x2jMcMH0fsnz+Wl8vJuG35Cdk52m/cdczUGGbJJLUNk7HaQxfXcoHATwb3gDZPfQ7GIPRAuECKgZDARBH6vPv1eLYom5Gtry/NiMJyIYkeI5BOySmi4jVGQTWqqiY6/XrZC0a9gzyi7HWRj82vMut8DxB7wAwgRUBKEYT4OarXonJA/aPuEntz2gavtyCZkkdiZO6GS7p9brxULKkSTmmyik10vG6Eos6LYnPsoux1kYyvkmKOSTQbiDYQIKHrCMh97WS3qJvhkokIaUCrCNvXWxPrAT3hzGirpgXn5YoUniCququM42fW37LFvTrVS1ZDLjcv5R9XtIBvbmOGDjMbsp8UkKtlkIL5AiICiJyzzsdvVosmK1iSdlcH6hdh2cFVZH0T739bcRtua25QrcJNj8/N6pdId2vgYGVF2O8jG9lbLWennnZV0/bSYRCGbDMQbCBFQ9ATdIl6GyWqRX5marsJV6axLZtXRzYKJwc/4F9X+ZRO9X1VcbVbzXsRElN0OsjFc6BWXhXOeuyCsPGFnk4F4g14zoOhhgoDn8a2Hpb1VgsI5TYh6vJj2dJFNRGtvm0wrPo+lYP1C3KDrMaKbjEXHYXpsouvFBBx/zr7z3C5lHxQvYkI0jrkTKrXfKwSye3rTnvyMJ6f4ddMzCICggUUElATO0uh819eg/P6q1Sf7P//e2tsmC7fFT5wya8uiaQnP4xaZ7nnTu6x8u2y8stdkr4vM/aLz+erBNnr1oNwdlExU0LW15bS/pTP7Wn2VufuAjeNnWw/RtuY27f4KieqeJiJ6cN44mt1QlXOsrx9qy/sc0aVKunCvgDCAEAFFiayKaCH9/m5Wn/379TUO/gvCNy8TT6KYArZ/vgy8bLw6VxV/zXhzv1t3kFOEEBEdOtVND21K0bpFSdWpyGFbc+4Evn7HETrS9gnNmzjCF/HnFtU9Peqq/JReUSp3Y7I6L/A5CkILlA4QIqDoUAXjFcLvzybUTy/0Wu9rzPBB1Di1Rtop11n9lP2/cWqNb2M3MdHzDeaevXu68WpaJp5MAihN3EH8vlmfGR5RqXrZMcjOycvvnaKX3ztF/7krTb9bdoNybEFiek/LjqN22BUFsxICIAJCBBQVumC8oNMN+QmVdws496UaB28NUHWe9XP1airI2KTGWzBM4D9rGkBp6w5SnTMioud3pY2EkO6c7G/ppE2706FZRkzvaVuxHYXsIFAaQIiAosLE9eLFpaFa+Ysm1P0tnbT2tsnUv1/fvO+YjkOXquvn6lU32TNeP9RGy3/9VvZvL2LIxl3Gx2wwRNlIumPYtOcYNZ88Rw9/fZK1eOV5q+VsqC4ak3tJJlhmN1QJY0yikB0ESgMIEVBUmJqp3aQb6twHsgm1f7++UveJyThM3CXbm0/5FivCT2p8/EBjspo2u6zNIcLWXZZMVNCGb6vdQaZZIPtbOun5XWnheyLx+vyutDAzZUrtUKP9BYnJvSQTLChKBsIEQgQUFUG5XkzcB0H1NjH5vnNF68U64dw3E0/JREXO5HW0vTtPiBC5N+W7vWYiFw8bo8057/jLeeHrMvHafPJcjrstWVseijXEbZaLSLCgKBkIEwgREApBpgoG8VA1cR/4IYJkVhcTdwnDrXVCZfExWW17MeV7vWaisfPnbMLIK+ngiU/yvjtv4giqq7zS6Lql0h1054zR9NVxn1BH93maUjs0FBESRD8Zdo1ZDRkIElAo+mQyGXEpvgjQ1dVF5eXl1NnZSUOGDAl7OMAnotxITEYq3UELn9qZ93rT0pnCWBG38SeqfYiyZj463S307z92xxRtNg2/PdPjI8q/hqyMfNCIzq3qvBFRXjyJk2RtOTV9nvGiu25RuW9t7kVbonKMIP7YzN+wiICCEuVGYipsrB1uy13rrC78dpk4cRNoyE84E0cOVu6bx1lMi4hodkOVcn8qTIWbbJJUnbcxwwcJRcjNX6zKqwGium5u7tugrH6y493efMrTfuL62wTxB0IEFJQoNxLTEbQf3U2MiRt3kGjCOXDinPW+nUGsj289LKzAqsN0Ba6aJFXnTXa/3fKlq63qr9jet0FaFmTH+/jWw9TzWW9BspcA8JOSFSIoZxwOUW4kZoJbawdDdd+5jTGxFUimGSWqfdtUYJVhswJXTZKNU2usz5vt/WZz3wZtWVClExcyewkAvyhJIQI/aHgEXVAsysjuO6c4WblgIo0ZPojeajlLU2qH0vgRg3MCB2VCxkYg6SaWf5heS7dfX6vcnm0FVpttiFbguklSJsb8ut9stuOXZUHUmdl5n5Rd1lfolgsqewmLNxAUJSdE4AcNn1JMFZTddye7/pbT/8NZiXXjrpacz/NVWt0KaF2Brim1Q7OTqeza2FRgtd2G6HXRmBuT1UZizK8Cdqbb8cOyoKvQy9xguvggW/HgpQQ/AG4pOSFSjH7QOK5UvLo44obsvuObkPEN2lTveRHQsoZ119aW04oX3s7+LYv7SCYqaO6EypzvivAzvmXlgok5wm1zqpWqhlxuNCG6ud8e2pTKuT5s8pXVLvHLCiOr0OuEXXuVOHMrHkT1WbB4A0FSckKk2PygWKnEg6DuLy8COpnIbVj36YXeHBFCpI77uH9uvVKI8BYLETbWClH3WDeZKybCnRchon2pfnu8i82m1ohpDM/R9m6pOJs/aaRv4iHqi7c4LsRALiUnRIopRgErlfggcy2IKpTa4IfAYSvgzfvyS5fz8D1Y5jRUCtNjJ4y8km6or6RUukN7L5oW0vIjc4Udg/M1XriLBA+/L91vz7nvjbtasqLBBNNrOmb4IKk4K7usr3L8NkR58YaFWHFQckKEqHhiFKK+UgG5iO67qiGXK2MBVMgsDm5XiG7iPh6YVy8UIgdPfJJtimcyOZhMKH5krvCs33GExgwflGOxUFkknMXkRLDXvSwQRKJV1sXZRDyKxm9DVBdvWIgVDyUpRIiKI0YhyiuVOFIIEy9/34nECV/xdHvzKWFQoihGwusKUWbhcOK8v3SBr0SXVuizG6qkwslkQvEjc0XEihfezrFYyH4/TuHnpnaJboGgC4wV3Z+yccxuqKKez3p9Ew9RXLxhIVY8BCpEXnvtNfrJT35Ce/fupY8//piampro1ltvDXKXJUVUVyqMuPhuU+mOvDLgqsJafh8TL074v1WTqnPC9rJC5AXM3AmVdP/c+rzOu6L7yzlJyUrOP771sLTomc2EIpsQ3TQKdMK7nERutMcWJbN/u/ntqcYkE5C6rCDZOIiIxo8YTGtvm0z9+/X15X6N2uINC7HiIVAh0t3dTVOmTKHvfOc71NjYGOSuSpYorlSIouG7NREN/DgZogk8rGPSPVjZhO1lJc6fg1cPttH9c+uN7y82SW3anVaOVRT8Kju+Ty/0Csdqmlpq0yiQKPc8mRy3Te0SVeCujYAUHT8/jpfePZHTi2bxrDqrKrJxIeoLMWBOoEJkwYIFtGDBgiB3ASj8lYqo8FLYvlsT0SAapxPnxOTlmLxaUXTuDzaRu10h2va4kSETdSrW7zhCD84bJ3yvf7/cgEvRNVVlh5hYapzw50l03Py1lJ0bm1RjUwFp0h05Cr+9QhLVhRiwI1IxIj09PdTT05P9u6urK8TRABNED8fxI+waqPmN6cNYF0fgnJjcWhv8sqKwB+7//cN7tPejs9nXnStAtytEP0zcOlHnBr4wl+ia6rJDnBO0SoiYnCdZXRERNqnGJuff6z1dzHETYS/EgHfEv+KQWL16NZWXl2f/1dbWhj0koED2cBSZ1IkK57vVZTQwXj8kD8rkJyY3k7Xs/KTSHcLPbt53TPge46V3T+SIkOsSQ+nmSSNzPrNywURqWjqTHrtjCjUtnUkrDAtYsdRWhq2JW3bO5zRUar87u6FKuf9UuoN+s6dF9FUpIusGv4/GZLXxeZLVFZFdL9N7UDY2/vybbg9xEyCORMoismrVKlq+fHn2766uLoiRCCN7OPbv1zdU363pClNUK+L6UUPp//zvLxoHBaqOya3JfWGymtY5AiPZeHlRszd9lhY+tTNvZe5mhejVxC075w/Mq6ehV/SX1uVg5zCZqDAqLc4zu6GKdrzflpPamhg2MO9zqXSH6+BNk7oiPLaCQHf+TbeHuAkQRyIlRMrKyqisrCzsYXgmLtkiRN7GKns4vn6ojdYtSobmuzV5GMtEwj98eZR0rLaTtVuTO5v0nGLENHPGC15M3KrsDdEk/uC8cXnpvPz+de4etn2+7kr6zF9zBJrIPWYTvKk69zLrn23AKvuOKhhYtD1RTyDETYC4ESkhUgxEIVvEFK9jTSYqaGGyOm+iaUq10p0zRofqu/Vrhcljc0xeBJHzHBLJJzzndmzPtY0INfns/EkjszEbTGTICm6Nukq/z+3Np4SvO7sDqwp6sWJlXoM3VfcEH1DrxEtvHNn22D39+qE22pxqzVbm9cMqBkBYBCpEPvnkEzp8+FKA2NGjR2n//v00bNgwSiTMey/EhThFrPs11q/VVwpXvFEIjrNdYXoxYcsmap0gUgkMdg5NslFkk6VsXDYilP/snIZKemBevXJ7PZ/1UjIhr+ehE3yqY2YixGQ7b7WcFb5uc3/KBLdu/5t2p5UBqybiTpSlQ0TZqrWi7QIQNwIVInv27KE5c+Zk/2bxH3fddRc999xzQe46FOIUsW47VtlD03aiiZLbyi8TtijG42v1ldoUTyL1ipr1EtGJEJmAkokN27oV/Ge3NbfRtuY2o+25QXXM/LHqUpun1A6ljbvyA11tgzeZm8wpLFTCVSWkjrZ35xWKEwlB2fWL03MGABMCFSKzZ8+mTCYT5C4CxXbSjErEusm4bcaqq19galmIotvKqwlbFuPRJDGZ8+jKictcDw/OG0ejrhokvcYqcWAzkZnEpshcKKrvqiZN1fdETxMmKJ949VBON+Als+po0bQEHW3v9sXytW5Rku6cMVpb1VUnHj+90KsVgqrrF5XnDAB+gRgRCW4mzShErJuO23SsJqtnE8tCnNxWNuhqkeiOURaEyMqJq3qJuM3YUU1ktqXS+dL4/PZkfHqhVziBH23vVrqrZOczmaig++fW0+QvlBNR7vkxvT9NFh28cLWpo0N08Tcms4I5xZnq+jVOrfH9ORMlSyUoPSBEBHiZNNlDj60SZzdUBTZOHttxmzRck9Vv4Fe0OstC1MzJfj14TVahumNUTZRuxG0q3UEfnRafb7Z90TZl7gJZfAQRSUWIc4wi18mKF97O+TsxbCClz/w1+7eqC7HofMpiVBiq+9OtpU72e1t722Th59feNpkWTUtIa4847yWd1cPPzBjV8UdRoIQxpiieh2ICQkSA10nT+UBnjb4K4YJwM27nA9qmRLetGThK5mRZqXA3DxpdjAKR2TGqJkqbSUd1DZ3igN/m+yfP5YkDJmJlAckTrx5MBz4+J9yXs9CaqAw7j1OEEF1Myb1t6hfohX3H8z770elu2rQ7na0HwsYqGrtJNpDb78pcUq1n/yoUeoumXQzQNxGXogBZ0We8Toqq4zeJYyk0Ybh3o+hSLjYgRAR4mTTDdEH4PW4ZbszAqoev32mkuu+Lro/uQaPar3NSf3H/cdrxfnveMXrFZNKRXUNnzY5Nu9P0VstZmlI7lBZNS2izclSunLtnjs4TL87v6dwMOl7Yd1xoGeFLtcuqt5osHGRj2958Kue7tvedTjzy7xMRbd53LPvZNVsO5IiQhclqoyq5tsjE1PbmU5FzpYbxbC1Wl3LUgBAR4CXWI2wXxJyGyhxzuddxO3HWb3CD6OHsJY3UzcrE5Dj5B43JfpOJCnrp3RM5IkQ1eQRh6pUdG6vZceuTf8pO6ht3tdB/7krTw1+fpLXmyH4Pi6Yl6M9HTmvTWr1Yvfa3dNLa2yZT69m/SnvFuIlR0X3m8a2HqeezXmlBtJULJtLshirhmJg7Vice2fuirCtVbR6/cNOkMMzMnDCerWE/z0sFCBEJbn2wYbkg+IfK3AmVdP/ces/jduJFhDCcD2evaaRuViam14E9aEz3K8ueEU0eQZl6Vffept3pPMvC/pZOen5XWrnNl949QclEhfT3sG5Rkt453kmHTl16YDtLrDPBpYoz0dG/X18adZX6ul03aqi0GSCPMyi2f7++0rGZFETzGjSqqqzLw1tpvKBLkZaJrDAzc3TP1iDEfZRcysUMhIgCNz7YMDJnRA+VVw+20f1z6423oYt1mDtB37zMBOfDwo80UtuViUlMB9GlB43pfk0/JxM2Y4YPysYQuEV178mCjjv+cl65TZ3YS6U7ckQI0aUS67xbZdb44XRt7VD606F22ps+a3hUlI1hUbH3o7N59VtEyKwA1yWGCsekK4i2csFEGjN8UI67ywa3biuvyPb74Lxx9NBNDUSUH2Qcds8a1f0dlLgP43leikCIBEChez34NUmL/NasNgP75+UHLjJBixCtNvxcmfDHyQflOR80pvs1/ZzsWq144e3seNxOakTiMutE8sJe9SMG06cXMlL3BhuzLHBRNZHyFhjmtrIRIexamEzYOveFygogG5OuIJrznt64q4WOtndb/T5s7l8/M/BUaeFE5k0C+ZijoJFl+gUZx4HePcEDIRIQfkS0m+LnJM27TpwFoojc/8BlJmhdZoBzXH6uTNhx6h64JtkLNuNTXRPnd1kMx++W3WB8TKoU1kXTEvSfu3LdM1WDB9CT2z7QbpcPwGVjVRXXksFvRwaLRyK6GMSp67XD+Nffv0ebl35V+J5OzMydUJlXEE1VE8SPCdDUQqdrmGeLrXVB1CRQFHNkc796Gbsba6Sf+wT+AiFSBARlPvTzBy7b1tfqK4XVKkX4vTIxeeDaZC+YjM904iG6aFFY8du36O+nJ3xJQ/3dshuyK9iKQWYihEguHlhxLS/xHzJuv742zwpTXzUozw3Esy99lh7alMrpXMzQiab759bT/XPrc66frLKt26qxIpzunQu9Gdq0J3+fN9T74xrl9+vWuiCLOdq0O10Qy4gTxHHEHwiRIiEI86GfP3DVtmxWG7YrE1kAm8kD1yYAVTc+5zjYxCNLfXWyac8x2rTnmNYlZioaF01L0KJpCfr2hl3afetg11RWZ2TEkDI62dVjvd0ls+qIKL82yKFT3TRr/HD65rVfoDHDB+WVdGfIrpFKBDqFu/P6qwrDybD9fXhpaugVt9YFWezMWy1nhUIkyIJgiOOIPxAiRYTf5kPRD9ytidjkYeH3w0rVjM7kgSv7DAv8NLFSHG2/2LLdOVE74ytM0yfX7zhCZZf1zfrw+fNkIxpT6Q5lTIiIWeOHS+ujfND2ifA7y28an5fey7s/nDgzvda90iz8zI732+m7N46nZOJiSXfZtpxZT85z5RTsLGtGdL+pxIHzN2AzAYriKbw0NQwC0/tIFjszpXZo3muFKAiGOI540ycT4a50XV1dVF5eTp2dnTRkyJCwh1OyPLQpJZxI3eC2Lb2tSEmlO2jhUzul78vcCU1LZ+asiFXbsKl5ItuP87ge+e93pWXNTcbA73PJrDqhG2nzvmN5beR1PHbHlKyAc14D1XE+dscUapxak9c2QHVOm5bOzHPJyLZLlH9vqrZjWqhOd935bZncm854CqKLJex/t+wG6bXQNTX0imrMpvcRf0zJ2nJq4mJEZOfS+TsDxYnN/A2LCFCSSnfkPei9RKSLrDY6N4ntiiqV7pCmqzJMAmV18Rw2NU942IrdGTR754zR9NVxn9Chk+fo5ffEFS9VY9BV62S4da3x127T7rRRaXv+e6qYElFFT9l2iSgbC8JfRyJ92XfZfWVb9E5niVTFU8iCYXVNDU1wK/p1PahEMUeyrBlV5VYIEcCAEAFKChGRrtoHkV0fEZtqkSaBsuyh/Js9LUJTtE3NEyfOyVQ0MSyedaXxcTjHwCZF1WQjygRSIXIN6M6zzJ3AB//aItruukVJunPG6JxGk7IJ0ClQZfeVqVB7flfa6Degi6cIIr5BVuCQyOz35BRXqnuJxRwB4AUIEaDExGfsNbZDtQ8bIWTTL4dt3ySuhr2vqiehe82Jc6KRWYOals6kMcMH0dYDJ7XWEX5/JoG4sgBTJ84eNbrtO3F2mdVlZDhRVfSUjYXBN5qUsXHXRUGp6k/TOLXGKLNJVwyO6OK5utAr9n6zeAq/4xtkBQ5fPdgmLUwoW1h4LcCnK4MPABGECNCgCzL1IxDNTdS7aLK3qVJpu+q0GaMsyPcGQdVP2Zh/tvWQUUApc0M4XTAm4s1ELLEKm050bi/Wg0Z0X8jcEM6aIaJy8LKxOMdk2zNF15/GpGPwvIkjlO+rrEbJ2vKcidwk28r0flX9DmTBvbL7QVeAT/dbR0bLJYLMHIo7ECJAi2zF5mdFQ9k+bB5kJiZ1UQ8e0weErGqpl8/KxiyaKJlVgOhS1sxL757ICQZcPKuO5k8aabQvPoNFJpYYOneM0xIiui/W3jZZ+D1RzZDrEkPphvrh2eMVxbow3JZJFxUw02VMMXghwaMSR7PGD6dffufL2vG5Ffm64m82jTF1BfhMfuvIaClM5lCcgRABRohWbH7Hj8hWhaYPMllwqaoHiekDQlW11MtnbWBddBmqVu02Vqw5DZX0wLz6HIHJT/w6q0Njsppaz/5VmnZLdLF5nWhcbNxO9qbP0t70WdrxfltOoKfo+ritsSEqYKbb5s1frKJ5E0do3RIqIbPj/XZKpTuUFpBPL/Qa1bmxyWJiPDCvnh6YJz5uHl3Atulv3cQFWqwEXYK+GIAQAa4pZEVD0weZyKQuK3DlpbOuTcaM6qFjs5oX9TcRwUzmfL2MVLojOx4n25rb6IF59cJt6zJJJl49mCaMHEybDQJQxwwfRI1Ta/L66ciqlxLl96sRxSfoJssls+ooQyQVZqbuNT6NNZXuyAmQtclMEk3gJiKCfU9UI+fOGaOtapKYToKqAnyoXqqnEAH/cQdCBLimEP5fW7+qTRdcWayD2866RBdjO0w/SyR/kMtSi3Ups85tJhMVeS4PVbAikX0myYGPz9GBj9WdcZ3jFzWJk7mSZIjiE0Spy/x9I3Mvyu4vlSWOFwKPbz2cl5mkEkcmAcYiPmj7hNa90iwU2+nTfxF+x4+aJIumJfIK8BVTrEeQ8RsoQa8HQgR4Ikj/rxu/qsmPXrfy5Ldh+iBRVSxln+UfeKJUWrby5lOLTVbMuowcVbCiLOX1+V1pWvt3U/JiC3TwE6DKWmTaf4f/nizllP3txPl+Kt2RVyLe1G8vEw38mNhv48HnU5Q+89fs566tLXeV8k1Eyv5Asg7CvLXGdNJVVaUtpliPoOM3ELCrB0IEeCYI/69bv6ruR2+SQiqawEzK0/+gSdw7Zu6ESqGQWPx5fISsqZ4zcPL9k+eU4xalt8omN15QyKwkjE17jlHFoAH0wLx6KyFiOh6nK4kXB9fWlkurzbo1bcsEnfP+UgmVJ14VW71kY3KKEKKL7iY+RsSv1bEqAJfIfUwU+1yxxXoUKn6jWEWcX0CIgEjixa+q+tHLtstSSGXbtjHT89w/t176wONxxrOYFmeTpbfKJrdbvjSSHphXn51o2b+FyWrpPtjDWWS94eMviMSrftl4/vjOx/RB2yc0tvJKYQDppt1pYXyCLjtEhE6IHm3vlpaYZ/EpMquSaEym97HIMuYGVQBuEDFRpkQ1dbWQ8RvFJuL8BEIERBKvflXZj172fZUIkW2TBSuqJjZmDVEFZPLI4jVk3KyIsRC5U1a88DYtTFbnTaiisvdO+NomzHrz0KZU3mdFq35Z3MTL753KKdrGr9IXTUvkNc9jx2FSy8LJ87vSyvdF2SpOZFVSGa1nc60fpvex14qzRPmBqCz7iQUrv3nktPB7XmKiTPC7j5SfIH4jGkCIgMDw8oAJyq/qx3ZT6Q7jgmOsrPbrh8xdGp9e6LXKpvnX379Hm5d+NTs2UcdfHtl7X6uvpK/UXSW0QPDH25Rqpa/UXSXdFjsGUZzB87vStGmPWJzxq2/VJG2zUl+z5YB0n0QX74P+/foqtyHrOisjmaigysEDqO3cpSqslYMHaC0QJiyZVUc3u7TSOXEbE2WC332k/AbxG9EAQgQocSsmvD5gUukOGj9iMK29bbK0VbtbvPhrbR7w7IEmahxIRHTdqKG096Ozea+z4zVlX/ospdId2q61Jjirs/LZNiKXhMpC8PqhtpzOss44A11TQrb6tmkgqMKmLL0MVjWWPzdO+NLlm3anc0QIEVHbufO0aXc6m4JsIjpHDB5Afz89QdVDB+b9HmytdPwxuYmJMsXPPlJBgfiN8IEQAVLciglRiqntypWvkcDw6yHhxl9rkjrrhLlMZA/jG8YNFwoR9jAUlYmX1euwmXwYqu7DonRYkRCRWQiSiaHKrs06y4IzBVeHiWhTbWfuhMps+flkokIYIOsMImbnhreKiSZrXcM70/GfPHdeW83XRiQTXQxwlpXO92tyVllXZFlaYdTXQPxGuECIxIRC+1HdBqypHoZuV65NqdbspBZWaWTbhzzRpcnvo9PiSXB2QxX1fNYrXXmKJoMMyd0qNjCzfmLYFdmxqNJdiUi4ShZZCOqrBlFKkkrK2r8vmpag/9yVFmbEOM+BTQNBFartsGBdVh5fNCa+KF4yUUEbvj1d+7usGDRAuE/W8I5tyyR9WfX7cePekTWecx5T49Qaq23yyKwrKusd4jNKDwiRGBCGH9VNwJruYeh15Upkb7rVTRQmAs+tD593TTi5btRQIhJbHpzl1XkxsG5Rkohyxcj1o4bSfk0QJR9PwPeoMSlDL1sl81VcRfElIn637AbatDtNb7WcpYpBA2hs5ZV510E2STt7Bvkl0tfvOJLtDySCuZP48cn2KROvoj41JudQ9fux7bcjymoSjdmPZ43oHnfee04Qn1GaQIhEnLD6FLgJWFM9DP1YuTr348bFwz9UTR+6OrP+/XPr6T/e+DBHHKjcKEREez86Swuf2pkTN2E6Hl6M7BG4d3hu/vx+kcVdmN5TsomXva7LDuJX4YumJbR9W0Rl+1892Eb3z6335RqasnFXC23c1WI0OcvEa7K2nJqW3SD8jvPc8llCbrpRE110v3zWm8krhCbKavLzWSMq3Me2IbtHVK4iUNyoQ8RB6OiCvYKCrUSduH0Yrr1tck6PDtv9mu7HieyhyoIRde+b7G/tbZPp2bun00vvnsgrSnZDvbpIGL9Pm/HIgl9VOO+XIO+pIMzqsnHJmv7ZXEOe2Q1V2vtPtg8nsjGnPhcAKvgsoVnjh2t/P7Lf60M3NdDYyiuNxigbs6qAm4g1Ww7Qwqd20vJfv0ULn9pJa7YcyHlfdi1kriJQ/ECIRJww89xXLphITUtn0mN3TKGmpTNdPwx1K17VfvkiW6aWFd1kazMZq45LFtNiU2zraHu31XjcCAbn/RLkPaUTkqKxs3oXsglaNq4/HWo33sdL757Ie+3a2vKcv9m9xe6/B+eNo5u/KJ4cddfAreVQdD/teL89W6tFda5kv1fT6y373KsH27TiSTV+Xri5WeSA4gaumYgTdp67bTS5X9H2bL+NU2vyeq6YoHv42k7GsuOSTSqilve2Y2Xv8WZuW8HA3y9B31M23VpNXCvJhLjqqKy3imlDuYe/PomIKKeDrnOfzI3lLLYm2wePbMy678ruJ7Yd5/Zk58ptOm4yUSHtJ2TqDjWNLUPKLHACIRID4vaj9TsVzs32dA9fvyZjlaBpnFqTE4D4x3dO5D3knfs0yS5gk49O5DR+7h6S3S+ie0oW9OkmGNSkW6tNTALL7uHR9VYhUk+OzsJvfAddIvl9QpQbVCxCFFjs1r3Jb4fILn7D9Bki6ydkKn5tBD5SZgGjTyaTyYQ9CBldXV1UXl5OnZ2dNGTIkLCHAyKI16wY00lWtHJnD3a+iinrniv77nWjhtIN44YLU2ad4yESZxc0LZ2ZFQ5sNV89dGC2vLibbqv8GFkQrkwIue3g6mTzvmPCrKLH7piSkzaqSp1uWjqTiEh7jUXncdb44bTj/Xz3jqiBoPM4ZOdEhvM66WqBEBHd9Nh2OnTKzP3GnyuviM41fz+LUJ0fk++D4sNm/i6IEHnyySfpJz/5CZ04cYKmTJlCTzzxBE2fPl37PQgRoKJQac2yiczJwmQ1fU1ggZB912QCdTtRz2mopAfm1Rtn4pgcH3+sOheBCapzw86HrOEdkd0Ex58HXVYTkd25co5Zt2/V+bK9Fqr92uLm2IjUIj0OFlwQDDbzd+CumU2bNtHy5ctp/fr19OUvf5l++tOf0vz586m5uZmqqhAlDdyhM+urVuK2rgaT4FDWd4XPTpEVM+OrcoomJxMzt+g8bGtuo23NbcIYBd6cn0p3aMut85hs0+T8ytwe7588R7/Z00LHz/5VaLEgsk/15F0TR9u7tUJE5Pqwra9jmxJrE4gsc/PY3t/s87J7lT823monOz4/LTWguAlciDz22GN077330re//W0iIlq/fj394Q9/oGeffZZWrlwZ9O5Lmqi23vYD1YQgMp3L3Cgmq3lT/7hpIS+i/AZyosnJJI5FNXGpmtGJrCVekG3T1lry//35IzrXc0H7OTepnm5iElglWIZpDIRuchcVR1Ntn/HgvHHZ/4vOgag9AotVEWFyDzjHJLK+iQijTDuIL4EKkfPnz9PevXtp1apV2df69u1LN954I73xxht5n+/p6aGenp7s311dXUEOr6gJu6tl0Mge2KI27ut3HJE+bE0C/kSCwAuyBnKih7cuyNBNyi3LxHFzPPVVg4TxC68faqMxwwdZrf5FYzARIW6CikUFttxcU9H35k6opPdPnpPGSIiQFUdLJsS9bhjpM3+RBtfKUsmJSChGTO4B57mWWd9EoEw7sCFQIdLe3k4XLlygESNG5Lw+YsQIOnjwYN7nV69eTY888kiQQyoJwqrGGgQyq45sIvnjO/n1InSYrN54QWDb6fbBeeNo1FWDlA3kZA9v1UpeN6HyAZlsYpFVt5w4cjAdOHEu5/M3T9KXb29KtUozW2Tn1009FNYl1waZKDcpq149dGDea+x7T7x6KNunRnQ9TRC5tWQiRBTX4vy+KvXX2SeHIfu88141sb6ZZC4BoCJS6burVq2i5cuXZ//u6uqi2traEEcUT9z0iYkiOquOqPS3bIWmwnT15hQEyUSFVX8VZ7YEX2mSyDwlVAQ/MTKurS3PESGzxg+n+hGDKZXukB7z/2ucTET5QbQq8aLDrwJqpsXxTGMY2PV01oThP7vihbfpaHu30JpoIz4enDeO2s71CDsOO3+XOnEgimth39fVJzG1pskye2Sfv39uPd0/t75o3cAgeAIVIsOHD6d+/frRyZMnc14/efIkjRw5Mu/zZWVlVFZWFuSQSoIwq7H6hYlVx4+S5F5Wb7pJTLQPmTn8RNffcjIWbF1pyUQFPXv3pW6wInG04/32rDBZPKsuL5iVr7PCo7t/0mf+kvea6vyauEdqKy6niVcPoYorBtDNk/KfGTxeYhhkhdhsglZlsHgOkRAxqXo7u6GK/uOND4Xvse8kE3ZF1Gxr6Yg+35isVt4zthRzXBuQE6gQGTBgAF133XW0detWuvXWW4mIqLe3l7Zu3Ur33XdfkLsuafwq1hUmsge9M8jPpoy6E12xLzfwrpv3T56jt1rO0pTaoTmreF3lTIZbVxoTRzrrBT/5L0xWG5fwFwmH60YNFU6AIvHgnGxEVi0iojFXXUGJq66gHe+3U0vH34iIaNOeY8I6JkSUFV9eYxj69xN3vdjefMp1dVtd0TrnNRaVomfWMtH5dQoBIrMias5zZ1ssceWCiXSy62/Z7W9OtVLVkMt9iT8r9rg2ICdw18zy5cvprrvuouuvv56mT59OP/3pT6m7uzubRQOCIW7VWHlkD3oW5CcL6Fsyq44yRHkP+5s9nguTlRoTAc4H6sZdLXS0vTvHjWOKG1caG+cHbZ9YfU8WR8Bzsutvwtf3SjoA81knphaLb1xbTY9vPZz3+vodR+jAx13StF4RshgG0TWV3XfOsbAsLBmq+031u5RZy9i2RDibK7LjuXPGaGFbhFS6I8995+z+bIKo4aIuINmkoKAsyDyOcW3AnsCFyKJFi6itrY1+9KMf0YkTJ+jaa6+lP/7xj3kBrMB/3KQresUv06rObC8SIc5ARtHD3u14ZCs10bHKXErO13gRJSuuZetK85qOqxM+brr+Pr71MPV81ps9X37E9NiIECKiyV8op/mTRlL/fn3p0wu91L9fX3poU0qYxm3iLlq/4wiVXSa2nDjrmzCxw8f9yH6XqtgunbtVZ02Q3Ru2k71N/JnbMem2C4qPggSr3nfffXDFlABuTKsq4cJWj7/Z0yL0rfM4zep+iTCZsHCap4kuHatJ7MD+lk5ae9tk6t+vb/a4q4ZcnpcS6nWctuisNW5jcthkJ/t+YthASp/5a85r+1vOutqXCGbNEO2HH2MyUZFjtfjodLfQMiPDWdtDVNND1cBRJTZU7laT4n6qe8Nmsrepo+JlTLr9geIiUlkzIL64SRk27bxKJA7y4wnioWUb02E6hv79+uZUnmSTH6u4ylJCTf3kfgTuOoWc02TOBNPrh9ylqLLxyc6NSBzYWj2crL1tMu37qIM27cmNk5GJEIbTjcSEbCrdIRQi1UMH5nWqdcZryGp6qIrp6WK7bDtAM4Ghuzdsfjem8WdexyTbLihOIEQAEXl3qQRZ+lr08BPFiLz07gnfH1w2D+mj7d3UOLXGqFCWbLuyiqts+7Lr44cIk5n5/YCNW9Zm3i+WzKqjo+3deSLELbJ7T5Su7QzcNJloRfe7SVdkG0uK6n0id5O9TBA5xynb50enu5Xp42tvm5zTvBGUBhAiwJdodduUYVvhwj/8iPI70wYR3CZLWRTFdDDXhq74mSyLQVYOXBZgqBunDSozv1fqqy5NVrI28zr4gNPGZDXVfl5A7bPeDHV0n6cptUNp/IjBVk3jnMgmPpPCZwxby5jofneKDVOroSytlt1bfFov667sR+q6bJyi+/HxrYezFWFFlhVnGjxfORYULwXpvusWdN8NHrcdN0XwDyNVd1Sv+zXtTOsX/KpUZjWQPThlFie31gfZeWL7WfdKczbtlYhofNUgel9Qmn3R9TU0svxyqh46kPr362sdEyGrW6Ear5tj5rsVyyrbysrn6zDt5Cu775ywe/CuZ9/UuphUXZhtfyN8AC5vNZR1iPaKrsP09uZTwnuKP3ai/MUF+xxcNPEjUt13QbTxswqrTcqw11onfhdt07mm+BXgygUThWmkKvcS/5qJ9cGmL41zP41Ta2jT7nROLRNeAFxbW+7ZhXH2L59mzekqAeMcr6xwGNHFrBNnPxWi/EJrqvNmIkKYNYCNSyQAvPT2GTN8EK3ZckArQhqT1cIGjUzI2vw2RRlNvOvSNEXbFtU4G6fWaN9n45HVvkHmTPEDIVLi+D2h22SreKl14lXIMFLpjmyAKMM020c20Zg+OE16fRDZ9aVxsmhaIqeYmo2LwZRtzW20rblNaoqXjVdWOGzUVYPooZsalNkluvgLXRwKc0mk0h1574kyXZxWBJ0LjBUfM7H41A67Ik+8OYWszW/TNFg5iEndbYwK/3oxVIQG7oAQKWFk/mNVwaco4bVom5faCqoHv+mD07TXh59Vctlk6qZvjMrtsX7HEWpaOpPmTxpJ//r792hf+mzeeE2CGZ3lykWrfpOicA/Mq6cH5tVL076PtncLLRGiKq/OTBdRszxnZVd2D7rtycNwVg82vfZe7zkv+LUosN1O1J9PwBwIkRJFt/IrRLllP/bhtl6Izi3Cl/TmkaWy8iW3VZg+eG0El+zhzL9uOiGZWGcYzNS+eelXtTE1i2fV5cUwXFtbbpzqLausy1ClfcuqeMqKlDk/I2qW59wfkdlkv2RWHc1uqBK6s1j1YPZ7MLn2JpllQabDqsZp42IyPV6Ugy8uIESKBJvVgazGAfMfb9qdDrzcspu6I36iM2XzJb2dDzlZhdHZ44fTY5/3+jDFZqLRCRCZi0n20DbJsmGZJGx8pu4X53hl15pnf0snpdIdeemgIuGwv6WTFl1fI4xxYROcrG+LzC10olNcvl60bRUygcmXfRe5hpzwwkeHyFKzvfkUEck76jL8sC7IxmnrcjG511EOvriAECkCbFcHqhWKLBuBva/K1LB5iPkZJOsGGxM1XxGSmc55vnHtF1yNxcaqIzrXKhdTR/f5vMmaHQ8fM/LHd07kCJkls+qELoympTPzUopVq22bYmvs+ptk1owsv1z4+pjhg5R9W2SYBO6a3jcygWliKXBi+3tg9xJ//tJn/hKadcEv1w0j7GcH8B8IkZjjZnUge5iKVp6677l9iMlcG4UKTBM9HOdOqKTJXygXmst1Io1IPnZRlVI3D0zRuZZ1r2XIJlf20HaKoEXTEnldbWW1Wp69e7qxALW5pioRwTO7oYp6PusVTnCqDIzGqTXGacdObCdPncA0zcAhUot9/j2ZxZPoUnde53cLYV2YP2lk1vWls87oQFBr8QEhEnPcrA5kKxSZyZq9b5J+avIQk7k2+BLZQQeiyapYioSITqTJJinZyt62poPsXOviGmSIHtr8OdelU5pacmQuEr5LMvvs+BGDtdtk5zuZqBBaHlST1ZotB3Luv+tGDZV2D2Y4Gyr6hS4Dh6ES+6L3ZOdPlL5bCOsCP8aez3o9bdtvCwsIHwiRmON2dSCbhEXIHsJuH2Ky77GW5oUMRBMFHdqItH+YXku3X18rdVnJJhlRNoYKP3rJMBqT1bS9+RRtbz6VXZ3KrC0ibFaeOheJSFytvW2ycFt8o0AGfw2ZoKqvGkSHHEXcrq0tp/dPnsvbp06ELJlV57sIYegaO25vPiUV++z//Huy80eU/9vUlWL3OrkHZXHxmjEHogWEiAFRThPzsjownYRlD2G3Ikj1vSgEotmINJkIITIXDybHp0r15d0TMlixMGd5+se3Hha6KdiYRPcDEeW1tpehEqsy+vfra3UfOlHFluxv6aT9LeLaKbUVA6ml41JTvMZkNd2gsVj59VxIJiro/ZPnhEJEFkCrO38y1xN/H8msMs5S7F4WAUFaXGxiq0C0gRDREIc0MT9XB4WojppMVEhTN8OuruicXJyl4t0cq43lQHd8qv0z9wSfNeOEpYsKS2hLYiWOtncL++Y4t6H7PbgRq+zc297TXvrktHT8VdhwjU2k/P79fi7ILG6qgNz/eOND6XuNU2voneOdeRYh0Xlk11hUit3rIgDxHMAECBEFUVidm+Ln6sBmW25EUCrdkVcDgqVuhvng0k0utsdqGgNAlH98otW2av/JRAVt+Pb0vGBTL4W2+OJibn4POgEnatbmnPxtYoa8uq+cWUP8hOy8F4J4LthYvJhVShVnlUp35IgQovz0aCfJREUg1gvEcwATIEQUIE0sGHS9J8J4cJlOLiYizTlp8imy/fv1pdcPtUl7qRCpBZFu/yJ3G0Ml5mTVdZ24/T2oBJTzvdcPtdHmVGvWdcSOm2/mxl7nxYmu4qoOVVl4571Q6OeC6PzJRCWLs3IzxqAWAYjnADogRBTArKjHjYlad15VD66g4nX8mlxk58O5jcapNdJeKkFa4WQWGtZ5VtXfhcj+98BfK5l1g73Od7Vdv+OItLHgya6/5YgTPjCV4Yz1UKVfm3TtZfdCEM8F3f3ntEpt3ndMKrrYGNyMMUjrBeI5gAoIEQUwK6pxO2mqzqssRoMo2HgdPyYXm/MhezAHvdp2xgMQ5dZ0MLG2mFhO3FR5lR23rLEg75YQiRA+24vF0ohcV0T6zr1OV5XfzwWT+09X5I3vUuxmjLBegDCAENGAH6YcL5Om6LyqhEbQ8Tq2D26RZUZ2PpxNzHQUwgrHWydMM2D4+hsLk9W0git9z1dcZazfcYTGDB8kvYZBWBn79+srtL6IXFdrthxQbou/F7w8F5wF7pzBsareOyaBuHzVWLdjhPUCFBoIEQNs4wJK5UfsddLkJ0SV0PDTUiC7Vl4bbsmOm29iphpLIa1wphamVLpDWM/CWSDLpCT7Wy1nha/LYoMak9U5qca2vH6oLcfdozo+20meSN0hWHb/yM6TqIje/pZO2rQ7TYumJbIWLBWi3wIvkkvl2RQGcZ0DojBuCBEfiEOKb9TRCQ2/LAW6a6UTnTrBpMqS4S04srH4ZYXTlQVXFcpypuuqJul/++NBmjqqwigzaErtUGGtDFVsUNWQy12l5F4/aqi0NoppEK6T//v79+gfvzJKmI3E4K/n3AmVdP/ceiurBs+KF96mo+3dRhV0Rb8FPJsKQ1zPc1TGDSHikTil+PqNn1YKndDww1Jgeq1kE7iq4R07Zl2lTPY53Vj44ETVBChCVimVfV927WSuFRlvHDlDbxw5o/0cK0h2tL07b7J2wgtBZmla8YK4EJmTxmQ1dfzlU9rW3EZ7JNVSRfemiZjdmz5Le9P523Rm8PDX89WDbfTqwTZtHIyO9TuO0LI5Y5Wf8bMFA7Ajruc5SuOGEPFIlFJ8C21i8zOewURoeLUUmFwr2QpB53pwHjPblmr172YsTlQrF9kDxvnawmS18Ls2IsSExLCB9PjfJ7PHxK4hC2blJ2sRIgHD8+C8cdKCbU5E96bOkqWCPbhVIsOPOJiO7vPC11UtBqL0bCpm4nqeozRuCBGPyNLovNY0sCUME5vf8QwmQsNLIJ2ur4aoD4ksyNKJ6Jh1mUEfnRY/BNgYdWZ81crFZOXdlGrNy4AxSWElIvpS9WB6p/Wc9nNEROkzfxW+ztfs0K3EVNU/iYiqhw7UHrfq3nTeex+0fUJPbvtAuS0nzswb1WdEcTD8+EZLrD8yt5ZIhDiDYUWg/IC/xLXMQ5TGDSHiEVlpZlUnW78ptIlNVrDLD0tMkBH7spUv66sh47mdHwpfV61Gicwyg5w4J0oTMSFbuZg+SBLDrqCmpTOtUlhVk6WMJ149RM/ePT1n3CJMytwnExXC3jorXnhbauVh1hI+PVzmjmucWkMXejPGFhK2LZXIEMXB8FkzzmvPC1iRVUgkrPj7i8/EQfkB/ylkgLmfRGncECIeiYKqLKSJzaRgV5TRraxFHPhYvPpXiRCGLjOIKHeiZJjcP7LPmLoaHt96mN4+3pkTUCl6MN0sEJo/f/2IsHaHiFcPtuWUFlf9Zkzci7K4EVnPHFWHYVVtE2fFV9m2nQ9u3u0k+gyRXmzLxL1O9Ivur/0tndKuxcA/4lrmISrjhhDxSBRUZaHEUJSCm7ygSgfmkbkrrhs11LcYlVFX5T8AdGJCd4/JJkUePkaDfzCxcfMPqSWzx+VVQlXxs62Hcnrh8G6hxmR1XqCsyr1oY3Fk51107/br20cbNMxXwmXbFD24k4n8nj9ufhsysaISMbL7q3+/vnnFAYH/BGnNDZIojBtCxAfCVpWFEkM6ywurN0FEeSv8oHD7wDcRaWtvm0zjRwwWCpG9H52lNVsOCHueyMZnKxh1ZnzRfvmmdyoR4kSUsaOKO7IVudua2+iuZ9/MqZS6MFlNX6uvzPaYUY2Jx2b/qiwhWSyI857mK/2m0h3afYbxcI+CdRZcJAq1OeIEhIhPhK0qCyGGVMFv/KT1+NbDgQfMmgToyh4IJhYHVh5c9jlRzxPnGETjsxWMyURFXi2Pns96815bmKyms5+nrjL41FgdTleezPpVdlnfrBCSnZfEsIHCIFW+XHtTqpWqhw6Uuj34MTl56d0Tea8tmVVHGSJfBLnonl78edfbKNRdEBEF66wtxThhR6U2R5zok8lkMmEPQkZXVxeVl5dTZ2cnDRkyJOzhlDSyIEsWQyBLm2xaOjOwgFnRPp37sxUqRHKT+7pXmo1jSpqWziQiko6PiIwtR7LjDALnudu875jS9cJ3wGWdhdl59GvMC5PVtG5Rkohys0FEwbJs/LLJzaT6K5H+npbtNyrEZXIvxgnb5LlUKtjM37CIAC2yIEvWVEzWkpwouJx0EzeRSTwLb8mSjXV2Q5WxEDna3i1Nz93efCon6+PxrYdpTkMlPTCvPmffbDKRbceUOQ2VeYGTNxsEVOrM+bwrh8dtXQ4eVkZeV+GVKL9TLRHluAvnTxqZ4+qSZf5kyCxrid9vVAjbOmtCscSb8USpNkecgBABWlRBcETqSSso/7TOH+73A0Fk9pb1QhkzfJBUQJzo/Btt2pMr3LY1t9G25jZpFocXHphXTw/Mq89bITsDKpk1w5nZYpJ5w6flOrGpiKpD1OdGBH9P6NyFsiJpJpVMnahqBoURNxUHinXCRpyOOyBEgBY35deJ/PVP2zaHC+KBYNILhY3hP974ULiNkeWXS7dvUjzNBr4tPI8o/sQ5UetSnfm0XB5R7YvGZDXdUF+ptEjwnOj8m/Yz/L0ms+I5V90rF0ykssv6Co/tyW0f5NXgkMGCiHlEQkhk/SpFinXCjmOcThSAEAFa3Py4+BbxRO59126awwX1QODN3vMnjcw2JHNms4gCMBuT1VoXz09eajYeC6sPIatzoQv+MglIZf/+51inMANHtII1LXj35yOnlYGqDN6CxJDVx0ilO+gHTXKR4xyz6nqwGhxvtZwVVjVlPL71MKXP/IW+Vl+ZHYtMCPHWr1KlmCfssLMo40hgwao//vGP6Q9/+APt37+fBgwYQGfPnrXeBoJVo4WqKqUfgaOyfXoJ/jKppOn2QcEfE1vtHm3vFgZ6PnbHFGqcWuPK9cLX3Vgyqy5H6G3anVYGcIowDUglMr8ONtdZt38V/PHL9i9CN2Ynj90xhcYMH2QVfMvSknXHVooBjDxxCawF9kQiWPX8+fN0++2304wZM+gXv/hFULsBBUQWBOdX4KhsGyKe3n6YJl49ROt3F41ZN1maPBxFx8RWu7JS43yZ7x80vS2t2uqETbrOolr8uGQFvtj5E33PNiBVt4K1vc5uzPAq14auP49ozETqeBaT8u08TalWOiNpUuck7vEQfhCHwNpCUqrCLDAh8sgjjxAR0XPPPRfULkBECDJwVLbtl987RS+/d8q6XolusjRd0auyKkQN5RqT1dnvsIfv/1s4WbnS5nvZqB7asvP0+qG2nJW583hMJljnNdKZnE0EqS7OR1aDhLGtuY0emFcv3Y8KluUlQtbh96V3T2TjSUwybhg73m+n60YNpb0fnZV+Ju7xEMBfijGd2RTEiARMKSjcIANHTSZLm7Q/1WTJtmWybd3Yv1ZfmbVgsMqhLMPG2Z+HFyxOTHrZMEQFvkRZPfzxqAI2ifKP040YkhUHk5WU17lB3DT7mzuhUihCnL/P+ZNGasu9s/3KMm6cjKu8UipEiiUeohSeb4WgWNOZTYmUEOnp6aGenp7s311dXSGOxjulpHCDDBxl2/7NnhZp0KCpmVs1WdpYbnQCyXkO+FgB5wNmxBBxFo3N+ZG5JGqHXSH8/PbmUznXSRWwySwCJsiuM5Fa4PHiRic8Vc3+ZMLu/rn5VhT+9ymrRMufL6Lc+/3F/cfzqsYSXQyy5TNvrh81lP7P//5iUUwupfR8C5piTWc2xUqIrFy5ktauXav8zIEDB2jChAmuBrN69eqsSyfulKLCVa2WvUaSs8/LhIipmduNKNL1guGbtTm3Z2uBIVK7EFTbMsUpOtjkoSpjb3PPiq6zrOCd7CGr6nyru1asCqvzOyKBIfp9inoKEYnPF9Gl+71xag09tCklFED7Wzpp1vjhWaGy56OzVuIuqpTi8y1IijWd2RQrIfK9732P7r77buVn6urqXA9m1apVtHz58uzfXV1dVFtb63p7YVLqCleE18A0mRXC1szN+/udxbzc9IJ59u7ptGl3mt5qOUtTaofmiAg3FhibzrKqfaTP/EVbC4NNHioXje09y19nNw9Z5ySvCtIVsW5Rku6cMTorEPkOw+yYRPCVaHlkk+26RUlKDLtCeP54a0kxTNh4vvlLMaczm2AlRCorK6my0q6Rlg1lZWVUVlYW2PYLSakr3KBwFtkicl+tMpmQF/PiYxY27zumnASdJuqNu1roaHu3MiBU94BRVeoUIYoPISKjGh1ElyYPmYvG6z3r5hyIAltV8D2DtjefyrNwOAWA7Jhu+dJIuuVLI+mtlrN0oTcjrGHys62HaMO38yvK2rYBiPMkg+eb/5Ry/ZHAYkTS6TSdOXOG0uk0Xbhwgfbv309EROPGjaMrr7wyqN1GhlJXuEEimphMg+acZc11cQumTfN0JmrVA0a0Al/xwts5YkaFScqqjjHDB2XPi6heSRD3bIbk18w29sCmLouzH43IumZS7XVbs7iirE2ab1QmbLfBpni+BUOppjMHJkR+9KMf0S9/+cvs38nkRd/ttm3baPbs2UHtNlKUssItJKYTl8mEZVv7xNREzT9gZDEFqn3J9uOV/3jjw5yxsIJcfhWBk51L0TWzjT2wFWJOAeClH44qvkWVhcSIQpyI12BTPN+AX9g5oy147rnnKJPJ5P0rFRHCYH5u/EiDQTZxpdId2s+JMKl9Ivq8bDsidCJEti/b/ZjCj6Up1ZrjlkqlO2jNlgO08KmdtPzXb9HCp3bSmi0HjLf/xKuHtJ9h10x2zMwVx2MjxEQrdtt4HIbqvM9uqNJ+X3SPFhLT342OYnu+pdId2XseFI5Ipe+CeBNGTQFTi4TJhOWm9onIRC1LAyUiaR8ak32JkO1//qSRnjrf/mzrIVdBmzypdIc0G4XHGePB8/jWw9TzWW/eiv31Q/ptPzhvnDSWyOQc27qrTF00YcaJINg0H6QjhweECCAi7yIirB+xqWCQfU7WOM3GB85M1GzyFmVpMExX8G7qrPDXjy+6JcqgmT1+OG0X1MBQiRCGyaRlY7Fg10iWucKLHxNRt2RWHT10U4P0fZ1oMCmvL8KkEmuYcSIINs0F6cjhAiECPIuIMH/EpoJB9jlVvQ5bHzg/eYrOgexB35ispm9ZTnZOREFuJpOhSITMnVBpZMV4/VAbNU6tUX7GdGJbMqsuL4tJxG/2XKwjk0xU0M+2il0+D84bR6OuGuRKNDhdc87vuwkidH6HF4VhB3Yi2DQXWIjCBUKkxPFDRIT9IzYVDG6C60wnIJugVX4CaExW02OfF+Ly+3yx7bEJXMfa2ybT+BGDjYRIU6qV7pwx2rObgrlOTDrcbtx1sbruwmS11GrjJqWbv85+XweTe09llQzC7Ylg00vAQhQuECIljh8iIgo/YlPBoPqcl4e9zTko5ARgk9pKdDF4UxZ3IhInJveJLjtldkOVdfaPzCUzd0JlZCdU1b2nskoG6fYs1XRRHliIwgVCpMTxQ0QUw4/Y68Pe9hwUYgJwU2OEXXeRu0IkREzvE1l2it/CYf6kkb5tq1CorJLs/6L3TM4bmtKZAwtReECIlDh+iYg4/4j9inGJ2jmQWRn+YXot3X59LT3y3+/mBK9eW1uurH1ic5/wE6BMsLBmdDbFwBiiBncrXnib/vjOCXpgXn3o598U01Rx/j3d8fllSSklMQMLUThAiADfJtC4/oj9jHGJ0jmQTf63X3+xfxOfQbO/pVNYMZQhuk9Ek5RsAtQJGef2PzrdrSwKxrJZvlJ3VZ7LZ1tzG21rFmct+YWfk7Mbq6TOEuWXuEZKKygEECKAiKI1gRaaKMS4BIHK2mXTEVfW90U0Sc2fNFI6AZoIXrb9VLpDKET4miCqgmRuM7d0IsPvyVlnlXRjsfRDXCOlFRQKCBFQ8iQTFQXrsVJoZJO/qfiSTbqySarsMrEwcPZ4MQ0qFk3AfE0QnVi0tWrxx8uXuw9qclaJNDcWSz/EddjZcKB0gBABJc+aLQfyeq2s0Kxw4+Q3d07+znHrVtqqSdc2y8WNdcnUgqKKLTFxYTgDcvntNKVas/fG4ll1NH7EYOF2/JicndYgvuOzrcXSj9ivYrUUgugBIQJKGtFkq6uPEVe/uWjcTUtnSid61YpYNhnNbqiins96PU2ATkwmYCZYnnj1UE5mj26//PlQleYnuihS1t42WfieX5Ozn/eW19ivYsiG80KcFhtxB0IElDS25ue4+s1V45ZVR1WtiFWTVDJRUfDsoWSigp69e7rx5CE6HyZF3Pr36xvY5BzEvSUTcqbnyWshtrgS18VGXIEQASWNrfnZrd887Ie1ybhFQamqSVc1SYUV/Gy6X1vXEuPTC700fsRgaY8iLwQdk8Gu7+uH2nJckbpJ1m0htrgS18VGnIEQAZ4Ie4K1xXay5XHjN/fzYe32fMvG9+mFXuUYdSti1SQV5XvDjSulviq3OuziWXXaXjt+jMkPt4+qwq6X7KJinLARpFt4IESAa+K2GnI72TqxFS66h7XNZO3lfMuCOle88Db9+cjpvMJgzjG6sW6EcW/YnEvbAmqiLsV+T7pBxWSYVNh1M8kW64SNIN3CAyECXBG31ZBuvDaTrY1wUT2s+W6zqsnaj/Mt6/ki69vidkIJ495wI3x0PXCILlWhPdreLexU7PekG0R1XhM3lJtJtlgn7FIP0g0DCBHgirithvwer6lwkT2U3zxymjbtyS0qppqsbcavsgyoCoCZjl0G2+9Hpwt7b3gRPoumJehoe7fUYnD79bXKbQQx6fodX6Mbo9tJtpgn7Ki1ayh2IESAK+K2GgprvDIXAC9CGLLJ2msBMt12vBZ0M+ny+9HpbmUJebfIRNr25lNG+zJJ/xVdx8ZkdXbfXo4p6Fga2dhvcBRqc0sxT9hhBVyXIn0ymUwm7EHI6OrqovLycurs7KQhQ4aEPRzAwU8+rP9HFEmlO+hnWw/RtubciaZQ402lO2h78yll/xQioqalM40zFPjxp9IdtPCpndptyrbjdkKU7VeG3/Eiqv3b7kt3DtxmnsgQVXJdtyhpvR0Tohw8DIoPm/kbFhHgmiBXQ34+NEWFq+6f6607q+34kokKra9eZ4XQnW9T941sO25XgLL9PjhvHBFRnvhyG6wrQxV4ahObYjIW9vryX7/lej/O/YmK6RFRIGLEywq/VERMqRxn1IAQiQFR/nG4fbipjsnvdFdR4SrWft4Nbscnc4vwjdxUqM63jfvJT7OzqsqqX8G6OlYumEgd3eeFLi8TF43NNfUr3ki2HV1l30ITt+w4t5TKcUYR88g1EAprthyghU/tpOW/fosWPrWT1mw5EPaQPKM6pk2708LAw1S6w9W+VJOGG2SBkSbjYyt3J6yRmx+Tjmz7QU9oqv2q6pf4eZ2JiEaWX658n/Vw4fdhe039ijdSfd7t/ekV/hx5ud/jRKkcZ1SBRSTCxC1F1gTVMfErZCduMy78DlL92dZDwtdNxxd0cF9YwYMqd48os0KWveMls2Z2Q5UwBmd2Q5VytWtr4fArWySZyO/6zAgj6Ft0joJs8hcl4pYFWGxAiESYYvxxqDIcVFkXbh/MfqYYptIdOcGubscncov46X4LK9pftl+RSLG1NpjuX3StifK76joFvRux6pfgY7EgXjKW3MDfb7IFQtBN/qJC3LIAiw0IkQhTjD8ON2P3+mD2a9KQiai5Eyo9jS+Kvmm/45J4kRJUDYr5k0ZS2WUXrS0s7mbzPnWqtOlYRO0B/Dg36xYl6c4ZowtmxbKxfATZ5C9KFHNNlDgAIRJhivHHITsmmVl97W2TadG0hC/79XreZCLKS+BrFN1vhRJGrLLpWy1naUrtUM/XmR93z2e9xhYPnVgN+pwUyorlxvLROLWmaGuFOCnmmihRB0Ik4hTjj0N2TCKB4ocI8YsghGHU3G+FFEbOyX3jrhY62t6tndxllhrduE2um0wM+HVOopD9JrvfdJaPUinuVSrHGTUgRGJAMf44RMcUB9Hl9xij5n4rlDByM7l7CTj1ct38OCdRcb+p7rdSsXyA6IH0XRApkokKapxaE+mHoJ9jDCvlVoZbYSRLjZVhm1atS680Gbfb6+ZFLKbSHbTulebIpIbq7rc4/P5A8QGLCCgpomAe54mSJciN+8nNat92ctdZJYKMp3K7bV3/nbDcb1G63wAgghABJURUzOMiouR+s5mo3MZPiCZ3VRM5PwJOvWC7bdF54Qkz+y1K9xsAECKgJIhidkqUMZ2ovHS+dU7urx9qo82pVtr8eT0NXiR6DTj1A5tte+0rBEApASECQqOQbpKoZacUC7JV/eNbD1PPZ71aixM79yZN5OLkUvCjr1ChiKK7EpQWECIgFB7alPKljbopUctOURGniUFkqWCYWpxsRGJcXAoyC85DNzWEOKp8ouyuBKUDhAgoOLwIIQreTRKX4nBxnBhWLphIZZf1FRakM7E4xUkk2hB1C05U3JVxEt4gGAJL3/3www/pnnvuoTFjxtDAgQNp7Nix9PDDD9P58+eD2iWIAal0h7DJF1HwHUdXLphITUtn0mN3TKGmpTNpRcQm+Dh3AJ3dUCV83URMRC2F2U+inA7rd2dqNxRjd3FgT2AWkYMHD1Jvby8988wzNG7cOHrnnXfo3nvvpe7ubnr00UeD2m3RUiyrBtVDrhArYGbaZ3UvonQ+4xzH4tXiFHXrQTEStiUqKhYZED6BCZFbbrmFbrnlluzfdXV11NzcTE8//TSEiCVxNNfLkD3kGpPVBXv4BHk+vQjGsCcGr3gVE3GJ/ygWwnZXxll4A38paIxIZ2cnDRs2TPp+T08P9fT0ZP/u6uoqxLAiTbGtGmT1Ix77vB160AR5Pr0KnLAnBlNUYgtiIl6EaYmKu/AG/lEwIXL48GF64oknlNaQ1atX0yOPPFKoIcWCYlw1hPnwC+p8+iVwou6iKCbrHLhIWOIxLsIbBI+1EFm5ciWtXbtW+ZkDBw7QhAkTsn8fP36cbrnlFrr99tvp3nvvlX5v1apVtHz58uzfXV1dVFtbazvEoqJYVw1hPPxS6Q766LRYiKjOp4m7xU+BE1WrQrFZ50D4RF14g8JgLUS+973v0d133638TF3dpQj41tZWmjNnDs2cOZP+/d//Xfm9srIyKisrsx1SUYNVgz+o+n6ozqepBaBQgjHMoGW/xFaxBF7bYHLMbs9L3M9nVIU3KBzWQqSyspIqKyuNPnv8+HGaM2cOXXfddbRhwwbq2xfNft2AVYM3ZH0/dFUubSwAhRCMYbtF/BBbYR9DGJgcs9vzUornExQfgSmD48eP0+zZsymRSNCjjz5KbW1tdOLECTpx4kRQuyxqolyPIOrIVvKjrlKLOts6C0HWKYlCjRGv9T6icAyFxuSY3Z6XUjyfoDgJLFj1lVdeocOHD9Phw4eppqYm571MJhPUbgHIw+1K3s33gjIzhx20zMz/8yeNFFrnCh1HExdMjtnteSnF8wmKk8CEyN13362NJQGgELh1m0QpPifMoGWd+d/UPfDphV7h9qMeeB10bZhCCmUAogh6zYCSwG2cTVTic8ISRbo4GdM4GlmwcNQDrwtRG6YYhDIAXoAQASWDW7dJVKL6wxBFOvO/iXtAFiy89rbJtGhawr/B+kwha8PEXSgD4AUIEQBiRKFFkc78b+IekImV/v2inUVX6NowcRfKALgl2k8CAECo6DJlTDJp4hrLENdxAxA3+mQinMLS1dVF5eXl1NnZSUOGDAl7OACULLqATd37fKzFkll1vqY3B0Vcxw1A2NjM3xAiAIRA3KthuiGuxxzXcYNgwX2hBkIERBb8eFENE4C4g9+wHpv5G8GqoGAU6scbZbGDxnEAxBv8hv0HQgQUhEL9eKO+UkE1TADiDX7D/oOsGVAQbPu2uCEOvTeQiQFAvMFv2H8gREBBCOLHm0p30OZ9x7JCoxBixyteG8cBAMIFv2H/gWsGFAS/y1GLXDDzJ40UfjZqKxVUwwQg3uA37C/ImgEFxY9A0lS6gxY+tTPv9aalM+mld0+g7gMAAIQMsmZAZPGjHLXKBYOVCvCbKGdhAVAMQIiA2KGLN0HvDeAXUc/CAqAYQLAqiB0IFgNBk0p30LpXmiOfhQVAMQCLCIglcMGAoOCtIDyoF+ENuLoAD4QIiC1wwQC/EdWi4YlaFlacgKsLiIBrBgAf4WubgHihqzkDF6B74lBwEIQDLCIg1kTJzIvVXvyRWTsenDeOZjdUhX6PxRmURgcyIERAbInSxI9GWGKiJBRNkBXee+imhhBHVRygNDqQASECYknUJn6s9vKJklC0AYHQweB3dWVQPECIgFgStYkfq71coiYUbUEgdDBA5AERCFYFsSRqEz9qm+QShwaEIBySiQpqnFpTsr8NkA8sIsCIqPn6o2jmxWrvElETigCA6IKmd0BLlH39URNI4BL8fYMGhACUDjbzN4QIUKLqdIuJH+iAUASgNEH3XeAbUQsKBfECQZ9yINIAuAiECFACXz8A/hNldycAhQZZMzGlUKXEkQ0CgL+g1DkAucAiEkMKvZpCNggA/gF3JwC5QIjEjLAKRcHXH08QhxA94O4EIBcIkZiB1RQwBXEI0SSKNXAACBMIkZiB1RQwIe4l1osduDsBuASCVWMGgkeBCSixHn1Q6hyAi8AiEkOwmgI6YDkDAMSFQC0i3/jGNyiRSNDll19OV199NX3rW9+i1tbWIHdZMmA1BVTAcgYAiAuBlnhft24dzZgxg66++mo6fvw4/fM//zMREe3cmV8yXARKvAPgDWTNAADCILK9Zv7rv/6Lbr31Vurp6aH+/ftrPw8hAgAAAMSPSPaaOXPmDP3qV7+imTNnSkVIT08P9fT0ZP/u6uoq1PAAAAAAEAKBZ82sWLGCBg0aRFdddRWl02l68cUXpZ9dvXo1lZeXZ//V1tYGPTwAAAAAhIi1EFm5ciX16dNH+e/gwYPZz3//+9+nVCpFL7/8MvXr14/uvPNOknmDVq1aRZ2dndl/LS0t7o8MAAAAAJHHOkakra2NTp8+rfxMXV0dDRgwIO/1Y8eOUW1tLe3cuZNmzJih3RdiRAAAAID4EWiMSGVlJVVWVroaWG9vLxFRThwIAAAAAEqXwIJV33zzTdq9ezfdcMMNVFFRQR988AH98Ic/pLFjxxpZQwAAAABQ/AQWrHrFFVfQ5s2bad68edTQ0ED33HMPXXPNNbRjxw4qKysLarcAAAAAiBGBWUQmT55Mr776alCbBwAAAEARgKZ3AAAAAAgNCBEAAAAAhAaECAAAAABCA0IEAAAAAKEBIQIAAACA0IAQAQAAAEBoQIgAAAAAIDQgRAAAAAAQGhAiAAAAAAgNCBEAAAAAhAaECAAAAABCA0IEAAAAAKEBIQIAAACA0IAQAQAAAEBoQIgAAAAAIDQgRAAAAAAQGhAiAAAAAAgNCBEAAAAAhAaECAAAAABCA0IEAAAAAKEBIQIAAACA0IAQAQAAAEBoQIgAAAAAIDQgRAAAAAAQGhAiAAAAAAgNCBEAAAAAhAaECAAAAABCA0IEAAAAAKEBIQIAAACA0IAQAQAAAEBoQIgAAAAAIDQgRAAAAAAQGhAiAAAAAAgNCBEAAAAAhAaECAAAAABCA0IEAAAAAKFRECHS09ND1157LfXp04f2799fiF0CAAAAIAYURIj8y7/8C1VXVxdiVwAAAACIEYELkS1bttDLL79Mjz76aNC7AgAAAEDMuCzIjZ88eZLuvfde+t3vfkdXXHGF9vM9PT3U09OT/burqyvI4QEAAAAgZAKziGQyGbr77rtp8eLFdP311xt9Z/Xq1VReXp79V1tbG9TwAAAAABABrIXIypUrqU+fPsp/Bw8epCeeeILOnTtHq1atMt72qlWrqLOzM/uvpaXFdngAAAAAiBF9MplMxuYLbW1tdPr0aeVn6urq6I477qD//u//pj59+mRfv3DhAvXr14/+8R//kX75y19q99XV1UXl5eXU2dlJQ4YMsRkmAAAAAELCZv62FiKmpNPpnBiP1tZWmj9/Pv32t7+lL3/5y1RTU6PdBoQIAAAAED9s5u/AglUTiUTO31deeSUREY0dO9ZIhAAAAACg+EFlVQAAAACERqDpu05Gjx5NAXmBAAAAABBTYBEBAAAAQGhAiAAAAAAgNArmmgEAFB+pdAcdbe+mMcMHUTJREfZwAAAxBEIEAOCKNVsO0PodR7J/L55VRysXTAxxRACAOALXDADAmlS6I0eEEBGt33GEUumOkEYEAIgrECIAAGuOtndbvQ4AADIgRAAA1owZPsjqdQAAkAEhAgCwJpmooMWz6nJeWzKrDgGrAABrEKwKAHDFygUTaf6kkciaAQB4AkIEAOCaZKICAgQA4Am4ZgAAAAAQGhAiAAAAAAgNCBEAAAAAhAaECAAAAABCA0IEAAAAAKEBIQIAAACA0IAQAQAAAEBoQIgAAAAAIDQgRAAAAAAQGhAiAAAAAAgNCBEAAAAAhEake81kMhkiIurq6gp5JAAAAAAwhc3bbB5XEWkhcu7cOSIiqq2tDXkkAAAAALDl3LlzVF5ervxMn4yJXAmJ3t5eam1tpcGDB1OfPn182WZXVxfV1tZSS0sLDRkyxJdtFhM4P3pwjtTg/OjBOVKD86Mn6ucok8nQuXPnqLq6mvr2VUeBRNoi0rdvX6qpqQlk20OGDInkxYsKOD96cI7U4PzowTlSg/OjJ8rnSGcJYSBYFQAAAAChASECAAAAgNAoOSFSVlZGDz/8MJWVlYU9lEiC86MH50gNzo8enCM1OD96iukcRTpYFQAAAADFTclZRAAAAAAQHSBEAAAAABAaECIAAAAACA0IEQAAAACERkkLkW984xuUSCTo8ssvp6uvvpq+9a1vUWtra9jDigwffvgh3XPPPTRmzBgaOHAgjR07lh5++GE6f/582EOLDD/+8Y9p5syZdMUVV9DQoUPDHk4kePLJJ2n06NF0+eWX05e//GXatWtX2EOKDK+99hp9/etfp+rqaurTpw/97ne/C3tIkWL16tU0bdo0Gjx4MFVVVdGtt95Kzc3NYQ8rUjz99NN0zTXXZAuZzZgxg7Zs2RL2sDxR0kJkzpw59Otf/5qam5vphRdeoA8++ID+7u/+LuxhRYaDBw9Sb28vPfPMM/Tuu+/SunXraP369fSDH/wg7KFFhvPnz9Ptt99OS5YsCXsokWDTpk20fPlyevjhh2nfvn00ZcoUmj9/Pp06dSrsoUWC7u5umjJlCj355JNhDyWS7Nixg5YtW0Z//vOf6ZVXXqFPP/2Ubr75Zuru7g57aJGhpqaG1qxZQ3v37qU9e/bQ3Llz6Zvf/Ca9++67YQ/NPRmQ5cUXX8z06dMnc/78+bCHEln+7d/+LTNmzJiwhxE5NmzYkCkvLw97GKEzffr0zLJly7J/X7hwIVNdXZ1ZvXp1iKOKJkSUaWpqCnsYkebUqVMZIsrs2LEj7KFEmoqKiszPf/7zsIfhmpK2iDg5c+YM/epXv6KZM2dS//79wx5OZOns7KRhw4aFPQwQQc6fP0979+6lG2+8Mfta37596cYbb6Q33ngjxJGBuNLZ2UlEhGeOhAsXLtDzzz9P3d3dNGPGjLCH45qSFyIrVqygQYMG0VVXXUXpdJpefPHFsIcUWQ4fPkxPPPEE/dM//VPYQwERpL29nS5cuEAjRozIeX3EiBF04sSJkEYF4kpvby9997vfpa9+9av0pS99KezhRIq3336brrzySiorK6PFixdTU1MTffGLXwx7WK4pOiGycuVK6tOnj/LfwYMHs5///ve/T6lUil5++WXq168f3XnnnZQp8mKztueIiOj48eN0yy230O2330733ntvSCMvDG7ODwDAX5YtW0bvvPMOPf/882EPJXI0NDTQ/v376c0336QlS5bQXXfdRe+9917Yw3JN0ZV4b2tro9OnTys/U1dXRwMGDMh7/dixY1RbW0s7d+6MtZlLh+05am1tpdmzZ9NXvvIVeu6556hv36LTrzm4uYeee+45+u53v0tnz54NeHTR5fz583TFFVfQb3/7W7r11luzr99111109uxZWBs5+vTpQ01NTTnnClzkvvvuoxdffJFee+01GjNmTNjDiTw33ngjjR07lp555pmwh+KKy8IegN9UVlZSZWWlq+/29vYSEVFPT4+fQ4ocNufo+PHjNGfOHLruuutow4YNRS9CiLzdQ6XMgAED6LrrrqOtW7dmJ9fe3l7aunUr3XfffeEODsSCTCZD999/PzU1NdH27dshQgzp7e2N9bxVdELElDfffJN2795NN9xwA1VUVNAHH3xAP/zhD2ns2LFFbQ2x4fjx4zR79mwaNWoUPfroo9TW1pZ9b+TIkSGOLDqk02k6c+YMpdNpunDhAu3fv5+IiMaNG0dXXnlluIMLgeXLl9Ndd91F119/PU2fPp1++tOfUnd3N337298Oe2iR4JNPPqHDhw9n/z569Cjt37+fhg0bRolEIsSRRYNly5bRxo0b6cUXX6TBgwdnY4vKy8tp4MCBIY8uGqxatYoWLFhAiUSCzp07Rxs3bqTt27fTSy+9FPbQ3BNu0k54/M///E9mzpw5mWHDhmXKysoyo0ePzixevDhz7NixsIcWGTZs2JAhIuE/cJG77rpLeH62bdsW9tBC44knnsgkEonMgAEDMtOnT8/8+c9/DntIkWHbtm3C++Wuu+4Ke2iRQPa82bBhQ9hDiwzf+c53MqNGjcoMGDAgU1lZmZk3b17m5ZdfDntYnii6GBEAAAAAxIfid/gDAAAAILJAiAAAAAAgNCBEAAAAABAaECIAAAAACA0IEQAAAACEBoQIAAAAAEIDQgQAAAAAoQEhAgAAAIDQgBABAAAAQGhAiAAAAAAgNCBEAAAAABAaECIAAAAACI3/HzwdiunChWczAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 查看特征的分布\n",
    "plt.scatter(x = features[:,0], y = features[:,1], s=10) # s为点的尺寸"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 生成batch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 208,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([100, 2]) torch.Size([100, 1])\n",
      "torch.Size([100, 2]) torch.Size([100, 1])\n",
      "torch.Size([100, 2]) torch.Size([100, 1])\n",
      "torch.Size([100, 2]) torch.Size([100, 1])\n",
      "torch.Size([100, 2]) torch.Size([100, 1])\n",
      "torch.Size([100, 2]) torch.Size([100, 1])\n",
      "torch.Size([100, 2]) torch.Size([100, 1])\n",
      "torch.Size([100, 2]) torch.Size([100, 1])\n",
      "torch.Size([100, 2]) torch.Size([100, 1])\n",
      "torch.Size([100, 2]) torch.Size([100, 1])\n"
     ]
    }
   ],
   "source": [
    "def data_iter(batch_size, features, labels):\n",
    "    num_features = len(features)            # 特征的数量\n",
    "    indices = list(range(num_features))     # 特征的下标数组\n",
    "\n",
    "    random.shuffle(indices)                 # 取每一条特征的下标，并打乱顺序\n",
    "\n",
    "    for i in range(0, num_features, batch_size):\n",
    "        batch_indices = torch.tensor(indices[i:min(i + batch_size, num_features)])  # 从乱序的下标数组中取batch_size个，如果不足，则取完为止\n",
    "        yield features[batch_indices], labels[batch_indices]                        # 生成一个iter\n",
    "\n",
    "batch_size = 100\n",
    "for X,y in data_iter(batch_size, features, labels):\n",
    "    print(X.shape, y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3 初始化模型参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 209,
   "metadata": {},
   "outputs": [],
   "source": [
    "w = torch.normal(0, 0.01, size=(2,1), requires_grad=True)\n",
    "b = torch.zeros(1, requires_grad=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.4 定义模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 210,
   "metadata": {},
   "outputs": [],
   "source": [
    "def linear_regression(X, w, b):\n",
    "    return torch.matmul(X, w) + b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.5 定义损失函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 211,
   "metadata": {},
   "outputs": [],
   "source": [
    "def squared_loss(y, label):\n",
    "    return torch.mean((y - label.reshape(y.shape))**2) / 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.6 定义优化算法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 212,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sgd(params, lr, batch_size):\n",
    "    with torch.no_grad():       # 在更新参数时，不需要计算梯度\n",
    "        for param in params:\n",
    "            param -= lr * param.grad / batch_size\n",
    "            param.grad.zero_()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. 这里为什么要除以batch_size？ 因为用batch训练时，比如将100个数据作为一个batch，计算出来的梯度是这100个数据梯度的总和。\n",
    "2. 最后为什么要`param.grad.zero_()`？ 因为在计算时，参数的梯度会保存到grad里，同时第二次计算的会累加到之前的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.7 开始训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 213,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch = 1, loss = 13.967720031738281\n",
      "epoch = 2, loss = 11.33918571472168\n",
      "epoch = 3, loss = 9.20586109161377\n",
      "epoch = 4, loss = 7.474114418029785\n",
      "epoch = 5, loss = 6.068473815917969\n",
      "epoch = 6, loss = 4.927373886108398\n",
      "epoch = 7, loss = 4.001051425933838\n",
      "epoch = 8, loss = 3.249041795730591\n",
      "epoch = 9, loss = 2.6384196281433105\n",
      "epoch = 10, loss = 2.142712354660034\n",
      "epoch = 11, loss = 1.7401831150054932\n",
      "epoch = 12, loss = 1.4133270978927612\n",
      "epoch = 13, loss = 1.147930383682251\n",
      "epoch = 14, loss = 0.9324186444282532\n",
      "epoch = 15, loss = 0.7573977112770081\n",
      "epoch = 16, loss = 0.6152527928352356\n",
      "epoch = 17, loss = 0.4998056888580322\n",
      "epoch = 18, loss = 0.40604478120803833\n",
      "epoch = 19, loss = 0.32988765835762024\n",
      "epoch = 20, loss = 0.2680289149284363\n"
     ]
    }
   ],
   "source": [
    "lr = 1\n",
    "num_epochs = 20\n",
    "net = linear_regression\n",
    "loss = squared_loss\n",
    "\n",
    "for epoch in range(num_epochs):\n",
    "    for x, y in data_iter(batch_size, features, labels):\n",
    "        l = loss(net(x, w, b), y)\n",
    "        l.sum().backward()          # 一个batch的所有损失求和，反向传播求梯度\n",
    "        sgd([w, b], lr, batch_size)\n",
    "    with torch.no_grad():           # 一个epoch结束，计算所有数据的损失均值\n",
    "        train_l = loss(net(features, w, b), labels)\n",
    "        print(f'epoch = {epoch + 1}, loss = {float(train_l.mean())}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. 线性回归的Pytorch实现"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.1 获取并生成数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'torch.Tensor'>\n",
      "[Features and Labels shape]:\t torch.Size([1000, 2]) torch.Size([1000, 1])\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import torch\n",
    "from torch.utils import data\n",
    "from d2l import torch as d2l\n",
    "\n",
    "w = torch.tensor([2, -3.4])\n",
    "b = 4.2\n",
    "features, labels = d2l.synthetic_data(w, b, 1000)\n",
    "\n",
    "print(\"[Features and Labels shape]:\\t\", features.shape, labels.shape)\n",
    "\n",
    "#\n",
    "#   功能：对数据集划分batch，每个batch做成一个迭代器 \n",
    "# #\n",
    "def load_data(data_arrays, batch_size, is_trian = True):\n",
    "    dataset = data.TensorDataset(*data_arrays)\n",
    "    return data.DataLoader(dataset, batch_size, shuffle=is_trian)   # 如果该数据集用于训练，就需要打乱以增加模型的泛化能力及防止过拟合\n",
    "\n",
    "batch_size = 10\n",
    "data_iter = iter(load_data((features, labels), batch_size))   # 获取迭代器对象\n",
    "\n",
    "# print('[data_iter example]:\\n', next(data_iter))  # data_iter的一个迭代元素为一个[10个数据，10个标签]的list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- `TensorDateset`：将多个张量（输入数据和标签）组合成一个Dataset对象，使该对象被`DataLoader`使用，用于批处理和迭代。\n",
    "- `TensorDataset(*data_arrays)`：\n",
    "  - `data_arrays`是一个元组，即`(features, labels)`，对元组使用`*`运算符意为依次取元组中的元素作为函数的参数，所以也可以写为`TensorDataset(features, labels)`\n",
    "  - 该函数的形参要求：各个张量的第一维（样本数）必须相同\n",
    "- `DataLoader`：将一个`Dataset`对象按照`batch_size`、`shuffle`生成一个**可迭代对象**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2 定义网络模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 215,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch import nn\n",
    "\n",
    "# torch是包，nn是torch的子包，网络层（如：Linear）是一个py文件\n",
    "net = nn.Sequential(nn.Linear(2, 1))    # linear全连接层（输入为2，输出为1），sequential层序列"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.3 初始化模型参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 216,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([0.])"
      ]
     },
     "execution_count": 216,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "net[0].weight.data.normal_(0, 0.01) # 用正态分布填充权重\n",
    "net[0].bias.data.fill_(0)           # 用0填充偏置项"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.4 损失函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 217,
   "metadata": {},
   "outputs": [],
   "source": [
    "loss = nn.MSELoss() # 实例化损失函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- `nn.MSELoss`是一个类"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.5 优化器"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 218,
   "metadata": {},
   "outputs": [],
   "source": [
    "trainer = torch.optim.SGD(net.parameters(), lr = 0.03)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.6 开始训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 219,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch: 0 ,loss: 0.000195\n",
      "epoch: 1 ,loss: 0.000107\n",
      "epoch: 2 ,loss: 0.000106\n",
      "epoch: 3 ,loss: 0.000106\n",
      "epoch: 4 ,loss: 0.000106\n",
      "epoch: 5 ,loss: 0.000108\n",
      "epoch: 6 ,loss: 0.000106\n",
      "epoch: 7 ,loss: 0.000105\n",
      "epoch: 8 ,loss: 0.000107\n",
      "epoch: 9 ,loss: 0.000106\n"
     ]
    }
   ],
   "source": [
    "# 设置超参数\n",
    "epochs =  10\n",
    "\n",
    "for epoch in range(epochs):\n",
    "    data_iter = iter(load_data((features, labels), batch_size)) # 在每个epoch之前重新加载\n",
    "    for x, y in data_iter:\n",
    "        l = loss(net(x), y)\n",
    "        trainer.zero_grad() # 每次反向传播计算梯度之前，清空保存的梯度\n",
    "        l.backward()\n",
    "        trainer.step()      # 更新参数\n",
    "    # 一轮参数更新完毕，用该参数输出features和labels的loss\n",
    "    l = loss(net(features), labels)\n",
    "    print('epoch:', epoch, \",loss:\", f\"{l:f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- `l = loss(net(x), y)`：`loss`是一个对象，一个对象为什么能像函数一样使用呢？\n",
    "  - 因为该对象具有`__call__`方法，可以像函数一样被调用，接收两个参数：\n",
    "    - 模型的输出\n",
    "    - 标签\n",
    "- `data_iter`为`DataLoader`对象，在循环时自动迭代，但如果迭代完成，下一次想要重新迭代，需要重新加载"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "torch",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
